Papers with neural abstractive summarization models
Improving Factuality in Clinical Abstractive Multi-Document Summarization by Guided Continued Pre-training (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods for enhancing the factual accuracy of abstractive summarization models are not effective in fact-critical domains like clinical document summarizing. |
| Approach: | They propose a guided continued pre-training stage for encoder-decoder models followed by supervised fine-tuning on summarization. |
| Outcome: | The proposed approach improves the quality and factuality of the summaries and achieves the best-published results on the clinical document summarization task. |
Factual Relation Discrimination for Factuality-oriented Abstractive Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing factuality-oriented abstractive summarization models only consider the integration of factual information and ignore the causes of factuual errors. |
| Approach: | They propose a factuality-oriented abstractive summarization model that can identify the causes of factual errors. |
| Outcome: | The proposed model outperforms state-of-the-art models in factual metrics. |
Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution (2021.acl-long)
Copied to clipboard
| Challenge: | Abstractive summarization models have made great strides in recent years, but little is known about how they actually form summaries and how to understand where their decisions come from. |
| Approach: | They propose a two-step method to interpret summarization model decisions by categorizing each decoder decision into one of several generation modes. |
| Outcome: | The proposed method can identify phrases the summarization model has memorized and determine where in the training pipeline this memorization happened, and study complex generation phenomena on a per-instance basis. |
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work suggests that the degree of hallucination depends on factual errors in training data. |
| Approach: | They propose a method to use training data to reduce hallucination by ensembling parameter variations in training data. |
| Outcome: | The proposed method improves on XSUM and CNN/DM datasets on human evaluations and factual metrics. |